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In this short paper, we shall provide a dynamical systems' proof of the famous Kraft- 
^ . McMillan inequality and its converse. Kraft-McMillan inequality is a basic result in in- 
OO i formation theory which gives a necessary and sufficient condition for the lengths of the 
codewords of a code to be uniquely decodable [H El E]. 
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1 Kraft- McMillan Inequality 



Given a binary prefix code set C for an alphabet set A, the codewords ci,C2, ■ ■ ■ ,cn with 
lengths li,l2, ■ ■ ■ Jn necessarily satisfy: 

^2-'»<l (1) 

i=l 

where = \A\, the cardinality of set A. A binary prefix code C is a set of binary 
codewords such that no codeword is a prefix of another. Prefix codes are known to be 
uniquely decodable and easy to decode. A famous example of prefix codes are the Huffman 
codes which have minimum redundancy. 

The Binary map 

Consider the binary map (Fig. [T]) T : [0, 1) [0, 1): 

X I— 2x, < X < — 

2 

1 

I— > 2x — 1, 2 — ^ ^ ^' 
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It is well known that the binary map is a non-linear chaotic dynamical system, which 
preserves the Lebesgue measure (ordinary length measure). We shall prove two simple lem- 
mas regarding the binary map which will be used to prove the Kraft-McMillan's inequality. 

Lemma 1: 

Given any sequence (or string) S of Os and Is of length m, there exists an unique interval 
on the binary map of length 2""^ such that all initial conditions in that interval will have 
S as the binary symbolic sequence corresponding to the first m iterations. 

Proof: 

Consider the given string S of length m as a binary prefix in [0, 1) (i.e. think of S as CS* in 
binary). The interval [0.5*0, 0.51), where the overline indicates infinite repetition, consists 
of all possible binary numbers in [0, 1) which have S as the desired prefix. All these binary 
numbers when fed as initial conditions to the binary map will yield S as the symbolic 
sequence in m iterations (this is because the binary map can be thought of as a shift map 
which spits out the leading bits of the binary representation of the initial condition). The 
length of this interval is O.iSl — 0.50 which is 2~™. □ 

Lemma 2: 

Two symbolic sequences 5*1 and <S'2 which are not prefixes of each other correspond to two 
disjoint intervals of lengths 2"™^ and 2"™^ respectively, where mi and m2 are the lengths 
of Si and 5*2 respectively. 

Proof: 

The proof is obvious. □ 

Proof of Kraft- McMillan Inequality 

Since Ci, C2, . . . , c^v with lengths li,l2, ■ ■ ■ Jn are prefix codes, using Lemma 1 and 2, these 
can be seen as symbolic sequences of disjoint intervals on [0,1) with lengths 2"'^, 2"'^^ . . . , 2~ 
respectively. Any collection of disjoint intervals on [0,1) necessarily satisfy Equation [TJ □ 

2 Converse of Kraft- McMillan inequality 

Given a set of codeword lengths that satisfy Equation [H there exists a uniquely decodable 
binary prefix code with these codeword lengths. 
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Figure 1: (a) Binary Map. (b) Symbolic sequences of length 1. (c) Symbolic sequences of 
length 2. (d) Symbolic sequences of length 3. 

Proof: 

Let /i, /2, . . . , Im be the specified distinct codeword lengths such that they satisfy Equa- 
tion [TJ Without loss of generality, let us assume that li < I2 < ■ ■ ■ < Im- Let there be Oi 
codewords of length li, 02 codewords of length I2 and so on up to aM codewords of length 
Im- Kraft- McMillan inequality can be re- written as: 

M M 

J]a,2-'><1, ^a, = iV. (2) 

i=l i=l 

where = \A\ as before. Let us determine the maximum number of codewords that can 
have a particular codeword length li while still satisfying Equation [21 If there are 2'* + 1 
or more codewords with length /j, then (2'' -|- 1)2~'* = 1 + 2~'' > 1 violating Equation [2J 
Thus there can at most be 2'' codewords of length /». 

Let us begin with li. We know that there are exactly 2'^ disjoint intervals with length 
2~'i on the binary map which have symbolic sequence of length li. Since the intervals 
are disjoint, the symbolic sequences are necessarily prefix codewords. We first assign the 
symbolic sequences of ai of these disjoint intervals as codewords. Once Oi disjoint intervals 
of length 2~'^ were used up, we have lost ai2^^~^^ intervals of length 2~^^. The number of 
available disjoint intervals of length 2"'^ is 2'^ — ai2^2~'^ If 02 < 2^^ — ai2'2~'^ then we can 
allocate disjoint intervals to 02 codewords of length I2. This requires 02 < 2'2(1 — ai2~'i), 
which reduces to ai2~'^ + 022"'^ < 1 which is necessarily true from Equation [2l Thus 
we can use the symbolic sequence of 02 disjoint intervals as prefix codewords (of length 
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Figure 2: Converse of Kraft-McMillan inequality, (a) Assigning intervals to codewords of 
lengths {3, 3, 2, 2} which satisfies the Kraft-McMillan inequality, (b) Assigning intervals 
to codewords of lengths {3, 3, 2, 1} which satisfies Kraft-McMillan inequality with equality. 
This is known as a complete code. 

12). This argument is repeated for 03 and so on until we have allocated unique disjoint 
intervals to all codewords (see example in Fig. [2]). We have thus proved the converse of 
Kraft-McMillan inequality by construction of prefix codewords using symbolic sequences 
of disjoint intervals on the binary map. □ 
The arguments above can be extended in a straightforward manner for ternary and 
higher bases. In the case of codewords of base-S, the S-ary dynamical system is used 
(x Bx mod 1 for all x G [0, 1)). 
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